CONVERGENCE OF WEIGHTED AVERAGES OF 
RELAXED PROJECTIONS 



RYSZARD SZWARC 

Abstract. The convergence of the algorithm for solving convex 
feasibility problem is studied by the method of sequential averaged 
and relaxed projections. Some results of H. H. Bauschke and J. M. 
Borwein are generalized by introducing new methods. Examples 
illustrating these generalizations are given. 



1. Introduction and preliminaries 

Many problems in applied mathematics deal with finding a point 
in the intersection of a family of convex sets in Euclidean or Hilbert 
space. The solution can be achieved in algorithmic way as a limit of 
composition of projection onto these convex sets. Due to its importance 
the problem has been studied heavily for many years. We refer to 
[T] where the reader can find an extensive account of theorems and 
literature related to the problem, as well as a general approach which 
captures the earlier methods and results. In the present work we will 
generalize, in case of finite dimension, one of the main theorems of pQ, 
concerning the convergence of algorithm. Firstly we will improve the 
estimate for the average of relaxed projections, secondly we will admit 
repetitive control, and lastly we will make use of perturbation theorem, 
which allows to ignore projections with small weight coefficients. The 
first generalization is valid also for infinite dimensional inner product 
spaces. All these was possible by applying new techniques and new 
proofs as well. 

For a closed convex set C C M. d let Pc denote the projection onto C. 
For x G M. d the symbol d(x, C) will denote the distance from x to C. 
Assume we are dealing with a fixed finite family of closed convex sets 
C\, C*2, . . . , Cn- For a sequence of relaxation parameters a±, . . . , ajy 



2000 Mathematics Subject Classification. Primary 49M45. 

Key words and phrases, convex set, weighted projections, relaxation. 

Supported by European Commission Marie Curie Host Fellowship for the Trans- 
fer of Knowledge "Harmonic Analysis, Nonlinear Analysis and Probability" MTKD- 
CT-2004-013389 and by MMSW Grant N201 054 32/4285. 

1 



2 RYSZARD SZWARC 

such that < «j < 2 and numbers Ai, . . . , Ajv such that < Aj < 1, 
J2iLi Ai = 1 we will consider the weighted averages 

N N 

A,{(1 - aj)I + a 3 P Cj } = I - J2 W -Pc,). 

3=1 j=i 

It is well known that every operator Pq. is firmly nonexpansive (see 
[U Facts 1.5]), thus these weighted averages are nonexpansive as well. 
Since the expressions depend only on the products \ai we will intro- 
duce the set 

N 




(3= (ft,...,/? 



A>0, ^A<2 



i=l 



AT 
3=1 

Remark. Observe that for any (3 G B there exist relaxation param- 
eters ai,..., ckjv an d average parameters Ai,...,Ajv such that /3j = 
Ajttj. Indeed, if J^fcli 4 > we may set a« = X^fcLi and Aj = 

Pi (^J2^ =1 Pkj ■ On the other hand if = for any z we take ojj = 

and Aj = 1/N for any z. Therefore every operator Qp is nonexpansive. 
Remark. All the results in this work remain valid if we replace Pq 
with a firmly nonexpansive mappings Tj such that Tj(c) = c for c G Cj 
(see |D p. 370]). 

2. Auxiliary results 

Proposition 1. For any x G W 1 and any c G G\ fl C 2 fl . . . fl Cm we 

have 



N 



\\Qp(x) - cf < \\x - cf - V(2 - ^W\x - Pc,(x)|| 2 



where k±, K2, ■ ■ ■ , Kn « r e any nonnegative numbers such that ^2j=i K j = 1- 

Remark. We set /3j/Kj = whenever (3j = 0. If /3j > and «j = we 
set Pj/nj = +oo. 

Proof. With no loss of generality we may assume that c = 0. Then by 
the convexity of each set Cj we have 

(P c Ax),x-P c Ax))>0. 
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Hence 



(2) \\Qpx\ 



JV 



\xf + 



\x\ 



JV 

X)^(x-p Cj (x)) 



AT 

Y,Pj{x-P Ci {x)) 

3=1 



JV 



-2j2Pj{x,x-P Cj (x)) 

2 JV 

-2^/? i iix-p Ci (x)ir 

JV 

-2^/3 i (P c .(x),x-P c .(x)) 



JV 



JV 



< lkf + X>(*-/fc,(:r)) 

i=l 

Let «x, K2, . . • , kjv satisfy the assumptions. Then by the convexity of 
the function x i— ► ||x|| 2 we obtain 



2^/3 i ||x-P Cj (x) 



(3) 



JV 



5>i(z-Pc,(s)) 



JV 

^/^-^(x-Pc^x)) 

N Pj 

K 



<Y,v\\*- p ^)\\ 2 - 

3=1 3 

Combining (1) and (2) concludes the proof. □ 
Remark. Setting 

Pi 



implies 

JV 

(4) \\Q p (x) - cf < \\x - cf - (2 - Ef =1 Pi) ~ P cM\ 2 

3=1 

the inequality obtained in [TJ Lemma 3.2(H)]. 

Theorem 1. Given a family of convex sets C*i, . . . , Cjy with nonempty 
intersection C. Let (3 G B and I C {1, 2, . . . , N}. Then for any x G M, d 
and any c G C we have 

\\Qp(x) — cf < \\x — c\\ 2 — min 2^ maxc? 2 (x, Q), 

16/ iEl 
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where 



Vi 



W (2 - Ef=i Pk) 



In particular the inequality holds when I is the set of active indices, 
i.e. 

(5) I = {i I 1 < i < N, Pi > 0} 

Proof. Fix i <E I and set 

§#7 i ^ *> 



On substituting theses values into the inequality of Proposition 1 we 
obtain 

1 1 (5/3 (x) — c|| 2 < \\x — c\\ 2 — Vi d 2 (x, Ci) < \\x — c|| 2 — min u k d 2 (x, Cj). 

fee/ 

Now maximizing with respect to i G / gives the conclusion. □ 

3. Main result 

Theorem 2. Fix a family of convex sets Ci, . . . , Cjv with nonempty 
intersection C. Given a sequence G £. Let /W denote toe set 
0/ active indices for (3^. Assume that every index i G {1,2,..., iV} 
occurs in for infinitely many n. Let n k be positive integers such 
that n k _i < n k and 

{1,2,..., N} C i" (nfc - l} U /( n fe-i +1 ) U . . . U J^- 1 ), 

i.e. every index occurs at least once for n such that rik-i < n < n k . 
For 

2/T (2-EL4 n) ) 



(n) 



' A (n) + 2-EL^ n) 

Ze* 

z/ (fc) = min{i/f ) I n fc _i < n < n k ,i G J (n) }. 

Assume that 

00 

$> (fc) = +00. 
fc=i 

T/ien /or any x^ G K d toe sequence x^ defined as 
is convergent to a point in C. 
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Remark. This result generalizes pO, Thm 3. 20(H)] (see also [1, Cor. 
3.25]) in two essential aspects. First of all it allows repetitve control 
while [U Thm 3.20] could afford only intermittent control, i.e. when 
the sequence is of the form = kp. Secondly the coefficients z/^ 
are smaller than /if 1 ' , introduces in PQ as 

nf ] = 2/f } 
For example let iV = 2 and 



N \ 
k=l J 



fjM = -, = 2-1 
n n 

The algorithm is then 1-intermittent, hence we can take n& = k and 
I k = {1,2}. Thus 

oo oo 

^2 min{/4 fe) , /4 fe) } < +oo, ^2 min { p i \ u 2 } = +°°- 

k=l k=l 

Therefore Theorem 3.20 of pQ does not apply while our Theorem 2 
does. 

Proof. With no loss of generality we may assume that G C. Fix 
u(°> G R d and let u {n) = Q^n^u^ -1 ^ for n > 1. By Theorem 1 we get 

(6) ||u (n) || 2 < ||u (n-1) f - min max d 2 (u^ n - l \ Q) 
Iterating (jSJ) leads to 

n 

(7) || M Wf < || ^C0> || 2 _ y min v (m) max rf 2 (u (m-l) ;C r.) 

' jgj(m) ig/( m ) 
m=l 

< ||« (0) || 2 - min maxd 2 ^- 1 '^.) 

0<m<n l<m<n 

Lemma 1. Given a family of convex sets C\, . . . , with nonempty 
intersection C and a sequence (3^ G B. Let denote the set of active 
indices for fi( n \ Assume that every index i G {1,2,..., iV} occurs in 
j(n) y or a ^ /east one n. Taen /or an?/ positive number R there exists a 
nondecreasing and positive function T]r : (0, +oo) — > (0, +oo) snc/i t/iat 

maxd 2 (n ( "- 1) ,C i ) > m(d(u (0) , C)) 

n>l 

/or any n^ turf/i ||w^|| < -R and ^ C. The function r]R is indepen- 
dent of the choice of the sequence (3^ n \ 
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Proof. Fix r > and consider the set 

B r , R = { M (0) G M d : ||w (0) || < R, d(u {0) ,C) > r}. 

The proof will be completed if we show that for any G B r . )R there 
exists a positive number rjn(r) such that 

maxti 2 ^"- 1 ),^) > rj R (r). 

n>l 

iei(») 

Suppose, by contradiction, that for any m G N there exist vectors 
uf\ G -Br, and a sequence /^"l G £>, satisfying the assumptions of 
Lemma 1, such that 



max dP{u^\ X \C^) < — 

n>l K ' m 

(m) 



where 



W(m) = Q /9 (»)(«(m) 1) )> n ^ L 



By compactness of we may assume that —> y and y G -B r ,R. 
Consider the sets 1^1. Some indices of {1, 2, ... , iV} occur for infinitely 
many m. Let Ai denote those indices. Clearly we may assume, even- 
tually by restricting to large values of m, that only indices of A\ may 
occur in and each index does so infinitely many m. Therefore, 
fixing n = 1 and taking the limit in (jSJ) when m — > oo yield y G Cj for 
any i G Ai. If Al = {1, 2, . . . , iV}, then y G C, which is a contradiction 
with y G -B r ,_R. Otherwise we have Ai C {1,2, .. . , iV}. Since for any 
fixed m every index of {1,2,..., N} occurs in at least for one n, 

there exists the least number l m such that ffe \ Ai ^ 0. Observe that 
Qp{y) — V f° r an Y /3 G £> such that the set of active indices I of (3 is 
contained in Ai- Therefore 

(9) 2/ = Q „(lm-l)Q o(Im-a) • • • Q„(2) Q„(l) (?/). 

P (m) P (m) P(m) P (m) 

Define 

( 10 ) «(m) := ^ = <5«C!m-l)<5g(!m-2) • • • Q g(2) Q Ml) (lijjjj) ) • 

Since the operators Q R {k) are nonexpansive we obtain 

(11) ^2).y)<d(< v i/) 

Hence Uf\ y. 

(in) *> 
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Consider now the sets ■ Each of these sets contains elements 
which do not belong to A\. Let A2 denote those indices outside A\ 
which occur for infinitely many values of m.. By restricting to large 
values of m we may assume that only indices of A\ and A2 may occur 
in ife and all indices of A2 occur for infinitely many values of m. Set 

n = l{m) + 1 in dHJ. Then since uP, = life" 1 * 1 we get 
(12) max d 2 (uf\,Ci) < — . 

j 6 r('m) m 

(m) 

Therefore for any i G A2 we have 

for infinitely many m. As u9\ tends to y, when m — ■> 00, we obtain 
that y E Ci for any «6i 2 . Therefore y G Cj for z G A\ U *4 2 - 

By repeating this argument at most N times we get that y G C, t 
for any i = 1, 2, . . . , N. Hence y G C which contradicts the fact that 

y e fi r , fl . □ 

Let's return to the proof of Theorem 2. With no loss of generality 
we may assume that G C. Fix R > and assume that ||s ( ' - ) || < R. 
Then since every operator Qp is nonexpansive we obtain ||£ <n ' ) || < -R 
for any n. Assume that 

{l,2,...,iV}c |J JW. 

Then combining Lemma 1 with = a^™*-*) and formula (]7j) yields 
\\x( nk) \\ 2 < \\x {nk -^\\ 2 - min v\ n) max c/ 2 ^" 1 ), C<) 

n k — l^ 71 — 71 k n k — l < ~- n — n k 

ie/(«) iei"(™) 

< 1^(^-0 1|2 _ ^(fc) ^(d^fa-i^C)) 

This implies that the series 



fc=i 



is convergent. Since the operators Q/3 are nonexpansive, the sequence 
d(x^ , C) is nonincreasing. Therefore, by assumptions made on the co- 
efficients h>\ n \ we obtain that 7] R (d(x^ n \ C)) —> 0. Hence d(x^ n \ C) A 0. 
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Since is bounded, it contains a convergent subsequence x^"" 1 -*. De- 
note its limit by c. Then c EC. By Proposition 1 the sequence ||a; (n ) — c|| 
is nonincreasing. Therefore, it tends to zero, i.e. x^ A c. □ 



4. Perturbation 

Proposition 2. Given a family of convex sets C\,..., with nonempty 
intersection C and a sequence (3^ E B. Assume that for any m E N 
and x E M. d the sequence 

Qp(n)Qp(n-l) ■ ■ ■ Qf3(m)(x) 

is convergent to an element of C as n — ► oo. Let a sequence (3^ E B 
satisfy 

oo N 
n=l i=l 

Then for any x E M. d the sequence 

Q/3(«)Qp(n-l) . . . Qp(l) (x) 

is convergent to an element of C as n — > oo. 
Proof. By ([1]) we have 

JV 

(13) d(Q pik) (y),Qp W (y)) < £ -$ k) \ \\P Cj y-y\\ 

N 

<^|/3f-^ fe) |%,C). 

Denote for simplicity 
and 

X ^ QmQm—1 • • • QliS^)' 
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d(x( n \ C) = rf(g n Q n _! . . . Q m+ i(^ (m) ), C) 

n 

< d(Q n ...Q k+1 Q k (x^),Q n ...Q k+1 Q k (x^ k - 1 ^ 

k=in+l 

+ d(Q n Q n . 1 ...Q m+1 (x^), C) 

n 

< ^(^ (M ).^(^" 1) )) + «r,-l---Qm + l(x H ),C) 

fc=m+l 
n N 

^ E El^f-^fl^" 1 ^) +d(QnQn-l...Q m+ l(x (m) ),C) 
k=m+l j=l 

n N 

< E E l$ fc) - $ fe) K^ c) + d(4Qn-i . . . g m+ i(^ (m) ), C) 

k=m+l j=l 

Now the conclusion follows from the assumptions. Indeed, we may 
assume that d(x, C) > as otherwise G C for any n. Let m 

be large so that 

oo N 

2-^ 2^\Pj ' 3 2d(x,C) 

k=m+l j=l V ' y 

Next let n be large so that 

d(Q n Qn-i-..Q m+ i(x im) ),C) <e/2. 

Thus d{x (jl \ C) < e for n large. Hence d(x^ n \ C) — > as n — > oo. This 
implies that :r( n ) tends to a point in C (see the end of the proof of 
Theorem 2). □ 

Corollary 1. Fix a family of convex sets C±, . . . , Cjv with nonempty 
intersection C. Given a sequence (3^ G B. Let denote the set of 
active indices for (3^ and let be a sequence of subsets of 1^ such 
that 

oo 

E E & <■**>■ 

n=l 

Assume that every index i G {1,2,..., N} occurs in for infinitely 
many n. Let n k be positive integers such that n k -i < n k and 

{1, 2, . . . , TV} C J ( " fc - l} U U . . . U J ( " fc_1) , 
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i.e. every index occurs at least once for n such that nk-i < n < n\~. 
For 



(n) 



let 

Vj^ 1 = min{z/f 1 ' ) | n^-i < n < nk,i G J^}. 

Assume that 



5>?> = +oo. 



Then for any G W 1 the sequence defined as 



x 



is convergent to a point in C. 
Proof. Define 



o if % g \ J (n) . 



N N 

(n) 



Clearly we have 

/c=i fc=i 

Hence z/f^ > vf^ for 2 G , which implies 

~(fc) > ^ Therefore, the 

sequence (3^ satisfies the assumptions of Theorem 2. Consequently 
the sequence 

Q[j(n)Qp(n-l) ■ ■ ■ Qp(m) (%) 

is convergent to an element of C as n — > oo. The sequences /3W and 
satisfy 

oo AT 

n=i i=i 

Thus applying Proposition 1 concludes the proof. □ 
Example. Consider N = 3 and 

(3f n) = 4, A 2n) = ! , /f n) =2-2, 



? (2n+l) ^_ i / o(2n+l) j_ ^(2n+l) O - - 
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The scheme is 1-intermittent, i.e. 1^ = {1,2,3} for any n. Observe 
that the assumptions of Theorem 2 are not satisfied. Now, consider 
this scheme as 2-intermittent and let 

J (2n) = {2,3} J (2n+1) = {1,3}. 

Then we can apply Corollary 1 to obtain that this scheme leads to the 
convergence of the algorithm. 

5. Intermittent control 

The assumptions of Theorem 2 depend on the behaviour of the co- 
efficients (3^ where i E I^ n \ i.e. those coefficients which are positive. 
Roughly the conclusion holds if these coefficients are not to small and 
the sums = YliLi flt^ do not approach the value 2 too fast. By 
Corollary 1 we can allow some small coefficients (3^ by using pertur- 
bation technique. However in special case of intermittent control and 
when the sums s^> stay away from 2 we can entirely liberate ourselves 
from assumptions on all positive coefficients fi\ n \ 

Theorem 3. Fix a family of convex sets C±, . . . , CV with nonempty 
intersection C. Given a sequence E B such that 

N 

S W = 2 - £ , 

i=i 

for some constant e > 0. Let denote the set of active indices for 
(3^ and let be a sequence of subsets of I^ n \ Assume that there is 
a positive integer p such that for any k we have 

{1,2,..., iV} C J^- 1 ^ U u . . . U J (fcp - 1} . 

Let 

vf = min{^ (n) | (k — l)p < n < kp, i E J (n) }. 



Assume that 



= +oo. 



k=i 



Then for any x^ E IR d the sequence x^ defined as 
a» =Q /3( „ ) (x (n - 1) ), n> 1 
is convergent to a point in C. 

Proof. First observe that since s^> < 2 — e we have 



pr < <> = — - < m 
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Therefore we can replace the coefficients v\ n ^ with when applying 
Theorem 2 and Corollary 1. 
Let Uk = kp. If for 



z/ (fc) = min{/5 l (n) | (k - l)p < n < kp, i G I {n) } 



we have 



fe=l 

we can apply Theorem 2 to get the conclusion. Thus it suffices to 
consider the case when 

oo 

(is) u{k) < +o °- 

fe=i 

Let 

A = {A; G N | (3n)(3i) (A: — l)p < n < kp, i G J (n) \ J (n) , v {k) = (3\ n) }. 
For every k G A choose rik and ik such that 

(A; - l)p <n k < kp, ik G J (n) \ J (n) , ^ (fc) = 0% k) . 
By (j!5p we have 

oo 

£/t fc) <+-- 

fceA 

Define the new coefficients (3^ by nullifying the coefficients (3^ for 
i = ik and n = n^, i.e. let 



(») 




if n = rifc, i — ik for some A; 
otherwise 



By construction the sums s^* 1 stay away from 2 since s^ h ' < s^ fc ^. More- 
over C I'-' 1 -', where J*™** denote the set of active indices for (3^ n \ By 
Corollary 1 the convergence of the algorithm for the new coefficients 
implies its convergence for the original ones. Thus we can restrict our- 
selves to the coefficients ft!- . Clearly for i G we have ft^ = ft\ . 
If the new coefficients satisfy ( !T4|) we are done by Theorem 2. If not, 
we can perform the same transformation as before. After at most pN 
iterations we will obtain a sequence to which we can apply Theorem 2 
and which differs from original sequence as in Corollary 1. □ 
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Example Let N = 3 and 





= 1, 






j_ 

n ' 


^(3n) 


1 

n 2 ' 


^n+l) 


1 

n> 




-1) 


= 1, 


^n+l) 


1 

n 2 ' 


^(3n+2) 


1 

n 2 ' 




-2) 


1 

n' 


^(3n+2) 


= 1. 



We have 1^ = {1,2,3}. Let 

j(3n) = j( 3n+l) = {2}? j( 3n + 2) = {1} 

(k) 

and p = 3. We have z/) = 1 for any fc. Hence all the assumptions of 
Theorem 3 are satisfied. 
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